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AMENDED CLAIMS 
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1. A method for controlling a computer using at least one video image of a 
plurality of video images, the method comprising: 

(a) capturing n video streams, n being an integer of at least two, the 
n video streams each comprising a plurality of video frames and 
each comprising an image of a user; 

(b) determining a location of an object in at least one of the n video 
streams; 

(c) controlling a program executing on the computer based on the 
location of the object; 

(d) combining the n video streams with a user interface stream 
generated by the computer operating system, thereby forming a 
composite video stream; and 

(e) displaying the composite video stream, 

2. The method of claim 1 wherein capturing n video streams includes 
receiving a live video signal of a user generated by a video camera. 

3. The method of claim 1 wherein capturing n video streams includes 
receiving a stored video signal from a video storage device. 

4. The method of claim 1 wherein determining the location an object in at 
least one of the n video streams includes: 

(a) searching for a predetermined color in one of the n video streams; 

(b) in response to locating the predetermined color, identifying an 
occurrence of the predetermined color having the largest area; 
and 

(c) determining coordinates of the center of the occurrence of the 
predetermined color having the largest area. 

5. The method of claim 1 wherein controlling a program executing on the 
computer based on the location of the object comprises: 

(a) analyzing motion of the object in successive video frames to 
determine presence of a control event; and 

(b) controlling the program based on the control event 
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6. The method of claim 5 wherein each of the n video streams comprises 
an image of a different computer user, the object comprises an object 
associated with a user's hand, and the control event comprises a pointer 
movement event. 

7. The method of claim 5 wherein each of the n video streams comprises 
an image of a different computer user, the object comprises an object 
located in a user's hand, and the control event comprises a mouse click 
event, 

8. The method of claim 1 wherein combining the n video streams with the 
user interface stream generated by the computer operating system 
includes horizontally reversing frames of the n video streams to produce 
a mirror image of the frames of the n video streams. 

9. The method of claim 1 wherein combining the n video streams with the 
user interface stream generated by the computer operating system 
includes transparently overlaying the user interface stream on the n 
video streams. 

10. The method of claim 1 wherein combining the n video streams with the 
user interface stream generated by the computer operating system 
includes transparently overlaying the n video streams on the user 
interface stream. 

11. The method of claim 1 wherein combining the n video streams with the 
user interface stream generated by the computer operating system 
includes: 

(a) adjusting a transparency level of at least one of the user interface 
stream and the n video streams; and 

(b) generating the composite stream from the user interface stream 
and the n video streams. 

12. The method of claim 11 wherein adjusting the transparency level 
Includes dynamically adjusting the transparency level. 

13. The method of claim 1 wherein displaying the composite video stream 
includes projecting the composite video stream. 
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14. The method of claim 1 wherein each of the n video streams comprises 
an (mage of a different user and wherein the program comprises a 
collaborative desktop application, 

15. The method of claim 14 wherein the collaborative desktop application 
allows each user to control his or her own mouse pointer on a shared 
desktop. 

16. The method of claim 1 wherein each of the n video streams comprises 
an image of a different user, wherein at least some of the users are In 
different locations and wherein the program comprises a distributed 
computer programming application. 

17. A method for combining a plurality of video images, each containing an 
image of a user, with a computer desktop interface, the method 
comprising: 

(a) capturing n video streams, n being an integer of at least two, each 
video stream comprising a plurality of frames and each 
comprising an image of a user; 

(b) transparently combining the n video streams with a computer 
desktop generated by the computer operating system, thereby 
forming a composite video stream; and 

(c) displaying the composite video stream, wherein the composite 
image Includes transparent images of the users displayed with 
the computer desktop. 

18. The method of claim 17 wherein capturing the n video streams includes 
receiving a live video signal generated by a video camera. 

19. The method of claim 17 wherein combining the n video streams with the 
user interface stream generated by the computer operating system 
includes horizontally reversing frames of the n video streams to produce 
a mirror Image of the frames of the n video streams. 

20. The method of claim 17 wherein combining the n video streams with the 
user Interface stream generated by the computer operating system 
Includes: 
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(a) adjusting a transparency level of at least one of the user interface 
stream and the n video streams; and 

(b) generating the composite stream from the user interface stream 
and the n video streams. 

21. The method of claim 20 wherein adjusting the transparency level 
includes dynamically adjusting the transparency level. 

22. The method of claim 17 wherein displaying the composite video stream 
Includes projecting the composite video stream. 

23. The method of claim 17 wherein displaying the composite video stream 
includes displaying the composite video stream on a non-projection 
computer display device. 

24. The method of claim 17 wherein displaying the composite video stream 
includes displaying a mirror image of each user with the desktop. 

25. The method of claim 17 comprising controlling objects on the desktop in 
response to movement of at least one of the user images. 

26. The method of claim 25 wherein controlling objects on the desktop 
includes moving objects on the desktop, 

27. The method of claim 25 wherein controlling objects on the desktop 
includes activating programs associated with objects on the desktop. 

28. The method of claim 17 wherein the desktop comprises the desktop of a 
computer local to at least one of the users, 

29. The method of claim 17 wherein the desktop comprises the desktop of a 
computer remote from at least one of the users. 

30. The method of claim 17 wherein each of the plurality of video streams 
includes an image of a different user. 

31. The method of claim 30 wherein the different users comprise 
collaborators in distributed computer programming task. 

32. The method of claim 30 comprising controlling desktop objects in 
response to movement of user images In any of the video streams, 

33. A computer-readable storage medium containing a set of computer- 
executable instructions, the set of Instructions comprising: 
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(a) n video stream capturing routines, n being an integer of at least 
two, each of the video stream capturing routines for capturing a 
different video stream, each video stream comprising a plurality of 
video frames gnd an Image of a user; 

(b) a video frame analysis routine for determining a location of an 
object in at least some of the plurality of video frames; 

(c) a driver for controlling a program executing on the computer 
based on the location of the object; 

(d) a video compositing routine for combining the n video streams 
with a user interface stream generated by the computer operating 
system, thereby forming a composite video stream; and 

(e) a video display routine for displaying the composite video stream. 
The computer-readable storage medium of claim 33 wherein the user 
interface driving routine comprises: 

(a) instructions for searching for a predetermined color in at least one 
of the n video streams; 

(b) instructions for identifying an occurrence of the predetermined 
color having a largest area; and 

(c) instructions for determining the coordinates of the center of the 
occurrence of the predetermined color having the largest area. 

The computer-readable storage medium of claim 33 wherein the video 
compositing routine comprises: 

(a) instructions for adjusting the transparency level of at least one of 
the user interface stream and the n video streams; and 

(b) instructions for generating the composite stream from the user 
interface stream and the n video streams. 

The computer-readable storage medium of claim 33 wherein the video 
compositing routine comprises instructions for horizontally reversing 
Images of the n video streams to produce a mirror image of the images 
of the n video streams. 

The computer-readable storage medium of claim 33 wherein each of the 
n video streams comprises an image of a different user. 
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38. The computer-readable storage medium of claim 37 wherein at least 
some of the users are in different locations. 

39. The computer-readable storage medium of claim 38 wherein the 
different users comprise collaborators in a distributed computer 
programming task* 

40. A computer program product comprising computer-executable 
instructions embodied in a computer-readable medium for performing 
steps comprising: 

(a) receiving n video images, n being an integer of at least two, each 
video image including an Image of a computer user; 

(b) combining the n video images with a computer desktop image; 

(c) displaying the combined image; 

(d) tracking a portion of at least one of the n video images in the 
combined image; and 

(e) manipulating objects in the desktop image based on the tracked 
portion. 

41 . The computer program product of claim 40 wherein manipulating objects 
includes highlighting the objects. 

42. The computer program product of claim 40 wherein manipulating objects 
includes moving the objects. 

43. The computer program product of claim 40 wherein manipulating objects 
includes activating programs associated with the objects. 

44. The computer program product of claim 40 wherein each video image 
includes an image of a different user. 

45. The computer program product of claim 44 wherein the different users 
comprise collaborators in a distributed computer programming task. 

46. A computer system comprising: 

(a) a display device; 

(b) n video cameras for producing n video streams, n being an 
integer of at least two, each video stream including an image of a 
user; and 
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(c) a processing unit operativeiy coupled to the display device and 
the n video cameras, wherein the processing unit is adapted to: 

(i) receive the n video streams, each video stream comprising 
a plurality of video frames; 

(ii) determine a location of a predetermined object associated 
with a user in at least one of the plurality of video frames; 
and 

(iii) control execution of a program based on the location of the 
object. 

47. The system of claim 46 wherein the n video cameras are each 
positioned to produce a video stream including an Image of a different 
user. 

48. The system of claim 47 wherein the different users comprise 
collaborators In a distributed computer programming task. 
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